RGW Object lookup path

How does the object lookup works in RGW?

When accessing S3 objects, REST APIs come to RGW with three parameters:

account information (access key in S3),
bucket name,
and object name (or key).

When accessing a user’s data, the user record is loaded from an object named <user_id> in pool default.rgw.meta with namespace users.uid This namespace will have the username and username.buckets objects:

#  rados -p kerala.rgw.meta --namespace users.uid ls
dashboard
kj
kj.buckets
dashboard.buckets

The list of buckets associated with a username is saved in the omap of the username.buckets object:

#  rados -p kerala.rgw.meta --namespace users.uid listomapkeys kj.buckets
kjbucket
test

Bucket ID or marker of buckets is stored in the pool default.rgw.meta with namespace root

# rados -p kerala.rgw.meta --namespace root ls
.bucket.meta.test:1847c1d2-a429-4a01-a65b-30eac3893e1e.245100.2
.bucket.meta.kjbucket:1847c1d2-a429-4a01-a65b-30eac3893e1e.115075.1
kjbucket
test

S3 objects are located in a pool named like default.rgw.buckets.data. RADOS object names are <marker>_<key>, where marker is the bucket ID and key is the s3 object name Example : 1847c1d2-a429-4a01-a65b-30eac3893e1e.115075.1_10mb_object

An RGW object may comprise multiple RADOS objects, the first of which is the HEAD that contains metadata including manifest, ACLs, content type, ETag, and user-defined metadata.

# for i in $(rados -p kerala.rgw.buckets.data listxattr 1847c1d2-a429-4a01-a65b-30eac3893e1e.115075.1_object1); do echo $i && rados -p kerala.rgw.buckets.data getxattr 1847c1d2-a429-4a01-a65b-30eac3893e1e.115075.1_10mb_object $i  | hexdump -C ; done
user.rgw.acl
00000000  02 02 6f 00 00 00 03 02  0c 00 00 00 02 00 00 00  |..o.............|
00000010  6b 6a 02 00 00 00 6b 6a  04 03 57 00 00 00 01 01  |kj....kj..W.....|
00000020  00 00 00 02 00 00 00 6b  6a 0f 00 00 00 01 00 00  |.......kj.......|
00000030  00 02 00 00 00 6b 6a 05  03 30 00 00 00 02 02 04  |.....kj..0......|
00000040  00 00 00 00 00 00 00 02  00 00 00 6b 6a 00 00 00  |...........kj...|
00000050  00 00 00 00 00 02 02 04  00 00 00 0f 00 00 00 02  |................|
00000060  00 00 00 6b 6a 00 00 00  00 00 00 00 00 00 00 00  |...kj...........|
00000070  00 00 00 00 00 0a                                 |......|
00000076
user.rgw.content_type
00000000  61 70 70 6c 69 63 61 74  69 6f 6e 2f 6f 63 74 65  |application/octe|
00000010  74 2d 73 74 72 65 61 6d  00 0a                    |t-stream..|
0000001a
user.rgw.etag
00000000  66 31 63 39 36 34 35 64  62 63 31 34 65 66 64 64  |f1c9645dbc14efdd|
00000010  63 37 64 38 61 33 32 32  36 38 35 66 32 36 65 62  |c7d8a322685f26eb|
00000020  0a                                                |.|
00000021
user.rgw.idtag
00000000  31 38 34 37 63 31 64 32  2d 61 34 32 39 2d 34 61  |1847c1d2-a429-4a|
00000010  30 31 2d 61 36 35 62 2d  33 30 65 61 63 33 38 39  |01-a65b-30eac389|
00000020  33 65 31 65 2e 32 34 35  31 30 30 2e 36 37 38 37  |3e1e.245100.6787|
00000030  37 36 39 33 34 33 39 35  37 35 39 37 38 36 33 00  |769343957597863.|
00000040  0a                                                |.|
00000041
user.rgw.manifest
00000000  08 06 61 01 00 00 00 00  a0 00 00 00 00 00 00 00  |..a.............|
00000010  00 00 00 06 06 90 00 00  00 0a 0a 73 00 00 00 08  |...........s....|
00000020  00 00 00 6b 6a 62 75 63  6b 65 74 2d 00 00 00 31  |...kjbucket-...1|
00000030  38 34 37 63 31 64 32 2d  61 34 32 39 2d 34 61 30  |847c1d2-a429-4a0|
00000040  31 2d 61 36 35 62 2d 33  30 65 61 63 33 38 39 33  |1-a65b-30eac3893|
00000050  65 31 65 2e 31 31 35 30  37 35 2e 31 2d 00 00 00  |e1e.115075.1-...|
00000060  31 38 34 37 63 31 64 32  2d 61 34 32 39 2d 34 61  |1847c1d2-a429-4a|
00000070  30 31 2d 61 36 35 62 2d  33 30 65 61 63 33 38 39  |01-a65b-30eac389|
00000080  33 65 31 65 2e 31 31 35  30 37 35 2e 31 00 00 00  |3e1e.115075.1...|
00000090  00 00 00 00 00 00 0b 00  00 00 31 30 6d 62 5f 6f  |..........10mb_o|
000000a0  62 6a 65 63 74 00 00 00  00 00 00 40 00 00 00 00  |bject......@....|
000000b0  00 00 00 40 00 00 00 00  00 21 00 00 00 2e 33 52  |...@.....!....3R|
000000c0  58 59 56 6e 55 77 7a 6c  4e 78 5a 34 62 79 69 45  |XYVnUwzlNxZ4byiE|
000000d0  36 36 50 67 62 30 35 39  33 66 79 64 48 5f 01 00  |66Pgb0593fydH_..|
000000e0  00 00 00 00 00 00 00 00  00 00 02 01 20 00 00 00  |............ ...|
000000f0  00 00 00 00 00 00 40 00  00 00 00 00 00 00 00 00  |......@.........|
*
00000110  00 00 11 00 00 00 64 65  66 61 75 6c 74 2d 70 6c  |......default-pl|
00000120  61 63 65 6d 65 6e 74 11  00 00 00 64 65 66 61 75  |acement....defau|
00000130  6c 74 2d 70 6c 61 63 65  6d 65 6e 74 00 00 00 00  |lt-placement....|
00000140  02 02 21 00 00 00 04 00  00 00 6e 6f 6e 65 03 01  |..!.......none..|
00000150  12 00 00 00 00 00 00 00  00 00 00 00 00 00 01 00  |................|
00000160  00 00 00 00 00 00 00 0a                           |........|
00000168
user.rgw.pg_ver
00000000  0b 00 00 00 00 00 00 00  0a                       |.........|
00000009
user.rgw.source_zone
00000000  49 88 fa a4 0a                                    |I....|
00000005
user.rgw.tail_tag
00000000  31 38 34 37 63 31 64 32  2d 61 34 32 39 2d 34 61  |1847c1d2-a429-4a|
00000010  30 31 2d 61 36 35 62 2d  33 30 65 61 63 33 38 39  |01-a65b-30eac389|
00000020  33 65 31 65 2e 32 34 35  31 30 30 2e 36 37 38 37  |3e1e.245100.6787|
00000030  37 36 39 33 34 33 39 35  37 35 39 37 38 36 33 00  |769343957597863.|
00000040  0a                                                |.|
00000041
user.rgw.x-amz-meta-s3cmd-attrs
00000000  61 74 69 6d 65 3a 31 37  34 32 35 36 34 39 36 35  |atime:1742564965|
00000010  2f 63 74 69 6d 65 3a 31  37 34 32 35 36 34 39 36  |/ctime:174256496|
00000020  35 2f 67 69 64 3a 30 2f  67 6e 61 6d 65 3a 72 6f  |5/gid:0/gname:ro|
00000030  6f 74 2f 6d 64 35 3a 66  31 63 39 36 34 35 64 62  |ot/md5:f1c9645db|
00000040  63 31 34 65 66 64 64 63  37 64 38 61 33 32 32 36  |c14efddc7d8a3226|
00000050  38 35 66 32 36 65 62 2f  6d 6f 64 65 3a 33 33 31  |85f26eb/mode:331|
00000060  38 38 2f 6d 74 69 6d 65  3a 31 37 34 32 35 36 34  |88/mtime:1742564|
00000070  39 36 35 2f 75 69 64 3a  30 2f 75 6e 61 6d 65 3a  |965/uid:0/uname:|
00000080  72 6f 6f 74 00 0a                                 |root..|
00000086

The manifest which is stored as xattr as show above, describes how each RGW object is laid out across RADOS objects. Another easy command to see the object manifest is :

# radosgw-admin object manifest --bucket kjbucket --object 10mb_object
{
    "size": 10485760,
    "objects": [
        {
            "index": 0,
            "part_id": 0,
            "stripe_id": 0,
            "offset": 0,
            "size": 4194304,
            "raw_obj": {
                "pool": "kerala.rgw.buckets.data",
                "oid": "1847c1d2-a429-4a01-a65b-30eac3893e1e.115075.1_10mb_object",
                "loc": ""
            }
        },
        {
            "index": 1,
            "part_id": 0,
            "stripe_id": 1,
            "offset": 4194304,
            "size": 4194304,
            "raw_obj": {
                "pool": "kerala.rgw.buckets.data",
                "oid": "1847c1d2-a429-4a01-a65b-30eac3893e1e.115075.1__shadow_.3RXYVnUwzlNxZ4byiE66Pgb0593fydH_1",
                "loc": ""
            }
        },
        {
            "index": 2,
            "part_id": 0,
            "stripe_id": 2,
            "offset": 8388608,
            "size": 4194304,
            "raw_obj": {
                "pool": "kerala.rgw.buckets.data",
                "oid": "1847c1d2-a429-4a01-a65b-30eac3893e1e.115075.1__shadow_.3RXYVnUwzlNxZ4byiE66Pgb0593fydH_2",
                "loc": ""
            }
        }
    ]
}

So how does a Multipart read works?

When a request comes, it goes to the HEAD object. The manifest stored as xattr in the HEAD object has the list of its shadow objects. Then RGW combines these objects.

How does the object lookup works in RGW?​

How does the object lookup works in RGW?